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Introduction 

Continuing professional development is an important 
activity for most teachers in the K-12 system, and it is even 
required of some teachers in order to maintain certification 
(Maldonado, 2002). Professional development programs 
are predicated on the assumption that the teachers who 
receive the training will in turn positively affect student 
outcomes. When investigating the efficacy of any professional 
development program, it is important to consider both the 
program’s duration (one-shot versus multiple sessions) as 
well as its characteristics (content-specific material, inquiry- 
based learning, collaborative grouping, or established 
learning communities) (Kennedy, 1998; Maldonado, 2002). 

An integral part of any professional development 
program is an evaluation of the program’s effectiveness. The 
importance of this evaluation component is grounded in 
the need to understand if the program is meeting its stated 
objectives and to make recommendations for subsequent 
improvements. However, the relationship between teacher 
professional development and student outcomes has not 
been firmly established in the literature. There are only a 
few studies that make the direct link between professional 
development and student outcomes, and these are generally 
linked to specific initiatives (Banks, Fickel, and Moon, 2004; 
Kennedy, 1998). Historically, most evaluations of professional 
development courses rely on participant self-reported ratings 
of the course’s quality or usefulness, which are collected upon 
completion of the program. Much less is known about how 
and to what degree the information is actually being used 
in the classroom. Other studies have also relied upon more 
easily obtainable indices of teacher preparation or quality such 
as licensure status, years of experience, and match between 
program of study and the course content actually taught. 


Although preliminary, some encouraging work 
has explored the link between professional development 
participation and student outcomes in the College Board’s 
Advanced Placement Program* (AP*). In a study that 
investigated the effects of AP Vertical Teams® training (College 
Board, 2002), some modest effects with regard to increased 
minority participation in AP Exams were reported. The modest 
nature of these results was attributed partially to varying levels 
of Vertical Team implementation. However, the study also 
highlights the importance of a continuing program of training 
marked by follow-up activity and teacher acculturation to the 
program. In a more recent study conducted in Florida and 
summarized below, a connection is made between teacher 
participation in AP professional development and levels of 
student participation in AP Examinations. 

AP Professional 
Development 

The primary goals of the AP professional development 
program are listed on the AP Central* Web site as including: 
...expanding professional development opportunities 
such as our workshops and Summer Institutes, 
providing teaching resources for AP courses, addressing 
equity by building partnerships with universities 
and other organizations, and advancing the field by 
continually learning more effective ways to support 
the AP community. (College Board, 2006, Professional 
Development section, para. 2) 

In particular, the two types of AP professional 
development opportunities available in Florida in the 
relevant time period were AP Summer Institutes and AP 
half-day workshops. 
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AP Summer Institutes are four- or five-day course- 
specific workshops offered through colleges and universities 
and are usually led by college professors and/or AP teachers 
who have substantial experience in the discipline. The topics 
that these opportunities cover include: 

• AP courses: goals, objectives, content, resources, 
bibliographies, and equipment 

• The AP Examination: how it is developed and graded 

• Syllabi, lesson plans, and assignments 

• How to refresh and improve existing AP courses 

• Recent changes in AP Course Descriptions 

• Strategies for teaching students in the Pre-AP* years 
( Endorsed AP Summer Institutes, 2006, p. 2) 

These events are most commonly attended by teachers 
new to the AP Program and focus primarily on providing 
content- and course-specific materials, as well as teaching 
strategies. 

Half-day workshops, on the other hand, tend to serve 
both new and more experienced AP teachers by providing 
information on updates to AP Course Descriptions and 
changes to AP Exams. Participants receive additional 
materials including “...course outlines; content-related 
handouts; and student samples, scoring guidelines and 
commentary for the most recent AP Exam free-response 
questions” ( K-12 Professional Development Catalog, 2006, 
p. 1). These workshops are typically four hours in length and 
have more emphasis on changes to and preparation for the 
AP Exam than they do on course content. 

Data 

The sources of data for this study include: 

• College Board data on AP professional development 
(1999-2004) 

• College Board data on AP Exam-taking patterns (2002 
and 2004) 

• Florida Department of Education (FL DOE) student- 
course data (2001-02 and 2003-04) 

• FL DOE teacher- course and demographic data (2001-02 
and 2003-04) 

• Market Data Retrieval (MDR) site capacity and school 
demographic data (2005) 

Given the many different sources of data and the 
complexity of individual-level data matching, this study 
performs analyses at the school level. It is clear that important 
variations exist at the student and teacher levels — in addition 
to the school level — and future studies would benefit from 


evaluating those effects in more detail. This study began with 
AP Exam-taking data for 333 and 351 Florida public schools 
for the 2002 and 2004 exam years, respectively. Data on AP 
professional development for the two years prior to the 2001- 
02 and 2003-04 school years were available for 309 and 343 
Florida public schools. All of these schools have associated 
with them the Florida Department of Education student 
and teacher data and the Market Data Retrieval school- and 
district-level data. After matching these various data sources 
the final samples include 317 and 327 Florida public schools 
for the 2001-02 and 2003-04 school years. The record of AP 
professional development has been limited to the two years 
prior to the beginning of the relevant school year because in 
that time period the teacher population appears to have been 
relatively stable. These data are summarized in Table 1. 


Table 1 


Summary Statistics — 2001-02 Cohort 
Sample Size: 317 Florida Public Schools 



Minimum 

Maximum 

Mean 

Indicator of 0.0%-15.9% of the School 
Population at or Below the Poverty Level 
(2005) 

0 

i 

0.46 

Indicator of 16.0% or More of the School 
Population at or Below the Poverty Level 
(2005) 

0 

i 

0.54 

Total Student Enrollment (2001-02) 

96 

5,423 

1,988.09 

Number of AP Teachers (2001-02) 

1 

34 

9.63 

Days of AP Summer Institute PD in the Three 
Years Prior to the 2001-02 Academic Year 

0.0 

94.5 

8.99 

Days of AP Workshop PD in the Three 
Years Prior to the 2001-02 Academic Year 

0.0 

31.5 

3.97 

Number of AP Exams Administered (2002) 

0 

2,107 

273.46 


Summary Statistics — 2003-04 Cohort 
Sample Size: 327 Florida Public Schools 



Minimum 

Maximum 

Mean 

Indicator of 0.0%-15.9% of the School 
Population at or Below the Poverty Level 
(2005) 

0 

i 

0.45 

Indicator of 16.0% or More of the School 
Population at or Below the Poverty Level 
(2005) 

0 

i 

0.55 

Total Student Enrollment (2003-04) 

70 

4,979 

1,982.24 

Number of AP Teachers (2003-04) 

1 

37 

10.48 

Days of AP Summer Institute PD in the Three 
Years Prior to the 2003-04 Academic Year 

0.0 

220.5 

31.13 

Days of AP Workshop PD in the Three 
Years Prior to the 2003-04 Academic Year 

0.0 

41.5 

7.07 

Number of AP Exams Administered (2004) 

0 

2,116 

349.81 
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Methodology 

The following models attempt to control for the following 
potentially confounding factors: (1) socioeconomic level of 
the school district; (2) school size; and (3) AP program size. 
The districtwide poverty-level data will serve as indicators 
of each school’s socioeconomic status, and the Florida DOE 
data on student enrollment and the number of AP teachers 
will control for school size and AP program size, respectively. 
Beyond these three factors, one expects that the stronger a 
school’s participation in AP professional development, the 
greater the effect on overall exam taking at that school. 

Upon further examination, it seems that within the two 
samples in question, AP Exam taking does not vary significantly 
with poverty level, average household income, or per-pupil 
expenditure. Given that Florida has made state funds available 
for AP Exam fees, the disincentive normally presented by the 
burden of those fees is removed. This policy has likely helped 
decrease the correlation between these socioeconomic controls, 
so much so that they have been excluded from the following 
ordinary least-squares regression models shown in Table 2. 

Results 

The model fit for both the 2001-02 and 2003-04 regression 
cohorts is quite strong with R-square values of 0.7232 and 
0.7753, respectively. The AP teacher count parameter is 
positive and significant at the 0.01 level, indicating that for 
each additional AP teacher a school employs, the school 
administers approximately 40 and 50 additional AP Exams 
for the 2001-02 and 2003-04 cohorts, respectively. This 
parameter accounts for by far the most variance for either 
cohort, with partial R-square values of 0.6937 and 0.7718 for 
the 2001-02 and 2003-04 cohorts, respectively. 

The total student enrollment parameter is significant 
at the 0.10 level and very slightly negative in the 2001-02 
cohort and is not significant at the 0.10 level in the 2003-04 
cohort. Regardless of the significance level, the effect size is so 
close to zero that the effect of school size after controlling for 
the number of AP teachers is effectively zero. With that being 
said, the slightly negative effect of enrollment could reflect 
the difference between those schools that have many AP 
teachers because the school has prioritized its AP program 
and those schools that have many AP teachers because the 
school is large. Having a large school in and of itself may be 
a less powerful predictor than having a school climate that 
supports academic rigor through advanced courses. 


Table 2 


Regression Output — 2001-02 Cohort 
Dependent Variable: Total AP Exams 
R-Square: 0.7232 


Variable 

Parameter 

Estimate 

Standard 

Error 

t Value 

p Value 

Intercept 

-112.36 *** 

21.66 

-5.19 

< 0.0001 

Number of AP Teachers (2001-02) 

43.46 *** 

2.13 

20.43 

< 0.0001 

Total Student Enrollment (2001-02) 

-0.02** 

0.01 

-1.97 

0.0499 

Days of AP Summer Institute PD 
in the Three Years Prior to the 
2001-02 Academic Year 

-1.77** 

0.80 

-2.21 

0.0278 

Days of AP Workshop PD in the 
Three Years Prior to the 2001-02 
Academic Year 

7.30 *** 

2.16 

3.38 

0.0008 


Regression Output — 2003-04 Cohort 
Dependent Variable: Total AP Exams 
R-Square: 0.7753 


Variable 

Parameter 

Estimate 

Standard 

Error 

t Value 

p Value 

Intercept 

-154.62 *** 

23.75 

-6.51 

< 0.0001 

Number of AP Teachers (2003-04) 

51.87 *** 

2.11 

24.56 

< 0.0001 

Total Student Enrollment (2003-04) 

-0.03* 

0.01 

-1.81 

0.0719 

Days of AP Summer Institute PD 
in the Three Years Prior to the 
2003-04 Academic Year 

-0.29 

0.45 

-0.65 

0.5152 

Days of AP Workshop PD in the 
Three Years Prior to the 2003-04 
Academic Year 

2.95* 

1.76 

1.67 

0.0953 


w *: Significant at the 0.01 level. 
** : Significant at the 0.05 level. 
* : Significant at the 0.10 level. 


The models in Table 2 assume that the number of 
days spent at an AP Summer Institute and the number 
spent at half-day AP professional development workshops 
have significantly different effects on AP Exam taking. The 
data confirm the earlier supposition that different program 
durations affect outcomes differently for both cohorts of 
data, though the results across cohorts do seem to diverge in 
the size and significance of the marginal effects. 

The effect of AP Summer Institutes appears to be 
significant at the 0.05 level and negative in 2001-02, and it 
does not appear to be significant at the 0.10 level in 2003-04. 
This is at odds with what one would expect, but there may be 
factors outside the scope of these analyses that explain this 
outcome. Examples of such potential factors are included in 
the discussion section. 

Half-day AP professional development workshops 
also seem to have slightly different effects on the two 
cohorts, with the 2001-02 parameter being significant at 
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the 0.01 level and positive and the 2003-04 parameter being 
positive and only significant at the 0.10 level. The effect size 
across the two cohorts ranges from adding an additional 
seven exams for every day spent in half-day AP professional 
development workshops for the 2001-02 cohort to only an 
additional three exams for each day for the 2003-04 cohort. 
One must keep in mind that the parameter associated with 
the 2003-04 cohort is not significant at the traditional 0.05 
confidence level, and such a difference in significance levels 
may indicate some unidentified sources of variation across 
the two cohorts. 

Discussion 

The limitations of the data and design used in this study — 
foremost of which are the facts that the data are aggregated 
to the school level and that they are limited to Florida 
public schools — do not allow us to make causal inferences. 
This study should, however, fuel future discussion and 
careful analysis of professional development programs 
and the direct effects that they have on student outcomes. 
In particular, studies utilizing multilevel modeling that 
are able to link students directly with teachers could 
separately capture classroom-, teacher-, and school-level 
variation would circumvent many difficulties encountered 
in this study. 

Difficulties aside, when considering counter- 
intuitive sign and significance level of the AP Summer 
Institute parameter, one must recall that the main population 
attending AP Summer Institutes are teachers who are new 
to teaching AP and, for that reason, students may not feel 
sufficiently confident in their abilities and may not choose 
to take the AP Exam. This parameter may therefore be 
inadvertently tapping variance attributable to AP teacher 
inexperience. However, including average AP teacher tenure 
into the model does not add to the explanatory power of 
the model or counteract the proposed inadvertent proxy 
of teacher inexperience. Perhaps if the Florida DOE data 
tracked not only how long the teacher had been employed at 
that particular school, but also the total length of his or her 
career or total tenure as an AP teacher, such a model would 
yield more precise results. 

Another possible and related explanation may be that 
evaluating the effect of teacher professional development 
on AP Exam taking at the school level may confound any 
teacher- or classroom-level effects. Yet another possible 
explanation is that there is a longer lag between the time 


a teacher can incorporate what he or she has learned in 
an AP Summer Institute than between the time a teacher 
can incorporate what he or she has learned in a half-day 
workshop. Finally, given that such a large portion of the 
variance in AP Exam taking is accounted for by the number 
of AP teachers a school has, the varying significance levels 
may simply be a statistical artifact. 

The fact that half-day workshops seem to be associated 
with a more direct and positive effect on exam-taking patterns 
within a school seems to make sense, because the content of 
the workshop tends to focus on changes to the AP Exam 
and exam strategies and readiness. These are more targeted 
workshops from which new and experienced AP teachers 
alike can directly and immediately benefit, whereas it may 
take a longer time for AP teachers to incorporate what they 
learned in an AP Summer Institute into their teaching 
practices and exam-preparation strategies. 

These outcomes are consistent with expectations that 
targeted, one-shot, exam-specific workshops are associated 
with greater increases in exam taking and lend themselves to 
immediate implementation more than the broader, multiple- 
day AP Summer Institutes. Even after controlling for by far 
the most prominent predictor of AP Exam volumes at a 
school — namely the number of AP teachers — schools with 
a record of participation in these professional development 
activities generally appear to be associated with greater 
exam volumes than otherwise similar schools. 


Brian Patterson is an assistant statistician at the College 
Board. 

Vytas Laitusis is an associate research scientist at the College 
Board. 

The authors thank Jane Delgado for her thoughtful comments 
on earlier drafts of this paper. 
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